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(57) Abstract: System actions such as presentation of information to a user via a remote control device are carried out in accordance 
1^ with user preferences based on, e.g., observed user behavior over time. The preferences may be learned by observation of user actions 

and corresponding system state information, and the preferences may be verified by the user. Particular actions may then be repeated 
JZl, under the control of the system, when appropriate conditions are met. Repetition may be automatic, senu-automatic or user initiated. 

In an illustrative embodiment, information is presented to a user via a remote control device based at least in part on the determined 
^5 preference. The information presented to the user via the remote control device may include, e.g., an alarm signal associated with 

a particular parameter specified by the user, a visible icon presented to the user on a display of the remote control device, or other 

types of information presented via other types of interfaces, such as audio, speech or tactile interfaces. 
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Method and apparatus for presentation of intelligent, adaptive alarms, icons and other 
information 



Field of the Invention 

The present invention relates generally to remote control devices and other 
types of information presentation devices, and more particularly to user interface 
simplification in such devices based on observed patterns. 

5 

Background of the Invention 

Remote control devices are commonly used for controlling the operation of 
audio-visual components such as a television (TV), video cassette recorder (VCR) or stereo. 
High-end remote control devices may be equipped with liquid crystal display (LCD) screens, 

10 e.g., a screen having a diagonal measurement of typically about six inches or less, for 
presenting information to a user. The presented information may include, e.g., context- 
sensitive component control menus which change depending on the activity and component 
being controlled. These high-end remote devices also include appropriate logic and 
processing circuitry, and may include a mechanism for receiving information signals from 

15 other devices. 

An example of a conventional display-eqviipped remote control device is 
described in U.S. Patent No. 5,410,367, entitled "Television Program Scheduler for 
Providing an Indication to a User that a Television Program is About to Start." This remote 
control device includes a display which provides information alerting a user that a particular 
20 broadcast program is about to begin. 

Another similar conventional device is described in U.S. Patent No. 5,710,605, 
entitled "Remote Control for Controlling Television and Videocassette Recorder with a 
Display for Allowing a User to Select Between Various Programming Schedules." This 
device assists a user in selecting paid programming. The device includes a display with a 
25 scrollable menu such that the user can determine what programming is currently available. 
The user can then order the programming from a provider via a modem connection between 
the device and a telephone line. 

Yet another display-equipped remote control device is described in PCT 
Application No. WO 98/43158 entitled "Two-Way Remote Control With Advertising 



wo 01/56269 PCT/EPOO/13230 

2 

Display." This device includes a display on which information, such as an advertisement, is 
presented to the user. The displayed information may be related to programming presented 
to the user via the TV. 

Another conventional system is described in PCT Application No. WO/34599 
5 entitled "System for Dual-Display Interaction with Integrated Television and Intemet 
Content.*' This system uses two or more physically independent displays for enhanced 
viewing of multiple broadcast signals, Intemet data or other local data such as information 
related to the state of a user's home appliances. The system includes a hand-held and portable 
remote control with a motion picture display. 

10 The above-described conventional display-equipped remote control devices 

and their associated systems have a nimiber of significant drawbacks. For example, these 
devices and systems are generally imable to provide more sophisticated functions such as 
controlling data collection and analysis operations specific to a particular user, i.e., the 
devices and systems are unable to '"learn" the intentions of a user and whether or not the user 

1 5 would like these intentions to be carried out automatically in the fiiture when faced with the 
same state information. In addition, none of the devices and systems is designed to provide 
automated user-specific intelligent alarms and icons driven adaptively by information firom 
multiple diverse information sources. 

A need therefore exists for improved information presentation techniques 

20 which overcome the limitations of the above-described conventional devices and systems. 

Summary of the Invention 

The invention provides methods and apparatus for carrying out system actions, 
such as presentation of information to a user via a remote control device, in accordance with 

25 user preferences based on, e.g., observed user behavior over time. For example, an 

illustrative embodiment of the invention provides an improved remote control device which 
is enable of displaying or otherwise presenting intelligent, adaptive alarms and icons based 
on learned preferences of a particular user. The preferences may be learned by observation of 
user actions and corresponding system state information, and the preferences may be verified 

30 by the user. Particular actions may then be repeated under the control of the system, when 
the appropriate conditions are met. Repetition may be automatic, semi-automatic or user 
initiated. 

In the illustrative embodiment of the invention, a remote control device 
operates in conjunction with, e.g., a computer, set-top box, television or other suitable 
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processor-based device to automatically determine the preference of a user with regard to 
presentation of a particular type of information available from a particular information 
source. The information is then presented to the user via the remote control device based at 
least in part on the determined preference. 
5 It should be noted that in other embodiments of the invention, preference 

determination, information presentation and other functions may be implemented at least in 
part in the remote control device. 

The information may come from one or more of, e.g., a network cormection, a 
telephone connection, a computer connection, a security system, a satellite system, a 
1 0 terrestrial broadcast system, a cable system, an information storage system or any other 

suitable source. The information can also be generated at least in part by the remote control 
device, e.g., via user interaction. Examples of information that may be presented to the user 
via the remote control device include an alarm signal associated with a particular parameter 
specified by the user, and/or a visible icon presented to the user on a display of the remote 
1 S control device. Information may be presented in many other forms, e.g., in the form of audio 
information, speech, tactile sensations such as vibrations, etc. 

By way of illustration, a processing system in accordance with the invention 
may be configiu-ed to detect when a movie likely to be of interest to a user v^ll be broadcast, 
based on information supplied by a television recommender program, and to set off an alarm 
20 via the remote control device notifying the user before the movie is about to begin. 

Advantageously, the invention allows a remote control device or other system 
device to provide information which is adaptively tailored to the particular habits and 
interests of a given user or set of users. By displaying infomiation, such as alarms and icons, 
in accordance with learned user-specific behavior, a remote control device in accordance 
25 with the invention is less likely to present irrelevant or redundant information to users. In 
addition, it can serve as a presentation point for a wide variety of different types of 
information, from many different sources. These and other features and advantages of the 
present invention will become more apparent from the accompanying drawings and the 
following detailed description. 



30 



Brief Description of the Drawings 

FIG. 1 is a block diagram illustrating an example of a processing system in 
which the invention may be implemented. 
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FIG. 2 illustrates the interaction between a processing device of the system of 
FIG. I and information storage elements including a filter/rules database and an event log. 

FIG. 3 is a data flow diagram showing the operation of the FIG. 1 system in an 
illustrative embodiment of the invention. 
5 FIG. 4 is a'process flow diagram corresponding to the data flow diagram of 

FIG. 3, 

Detailed Description of the Invention 

FIG. 1 shows an example of a processing system 100 in which the present 

10 invention may be implemented. The system 100 in this embodiment includes one or more 
controllable devices 102, a processor-based device 104, a remote control device 106, and a 
set of communication and data devices 108. The communication and data devices 108 may 
include devices that are data input only, controllable only, or both data input and controllable. 
It should be noted that the processor-based device 104 may be implemented at least in part 

15 within the remote control device 106, e.g., the devices 104 and 106 may be physically 
combined in whole or in part. 

The processor-based device 104 includes a processor 1 10, a memory 1 12, a 
communications unit 114, and an adaptive engine 116. The processor-based device 104 may 
represent, e.g., a set-top box, a computer, a palm-top compixting device, as well as portions or 

20 combinations of any of these and other processor-based devices. 

The processor 110 may represent, e.g., a microprocessor, digital signal 
processor, central processing unit, computer, circuit card, application-specific integrated 
circuit (ASIC), field programmable gate array (FPGA), or microcontroller, as well as 
portions or combinations of these and other types of processing devices. 

25 The memory 1 1 2 may represent an electronic memory, an optical or magnetic 

disk-based memory, a tape-based memory, as well as portions or combinations of these and 
other types of memories. 

The remote control device 106 is equipped with a display 117, which may be, 
e.g., a conventional LCD display, of any desired size practical for use in conjunction with the 

30 device 106. 

Input to the remote control device 106 may be acquired through responses to 
traditional graphical modalities (such as buttons, sliders, menus, fill-in boxes, etc.), but the 
input may also employ handwriting recognition, shorthand-style pen gestures, speech 
recognition, or visual gesture recognition of facial or body expressions. 
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In accordance with the invention, the remote control device 106 delivers 
control data and filter criteria to the processor-based device 104, and receives filtered data 
from the processor-based device 104 for presentation to a user via the display 11 7 of the 
remote control device 106. The filter criteria and filtered data are determined in accordance 
5 with an automated learning process that will be described in greater detail below in 

conjunction with the data flow and process flow diagrams of FIGS. 3 and 4, respectively. 

Although the display 1 1 7 is shown as incorporated into the remote control 
device 106 in the illustrative embodiment of FIG. 1, this is by way of example only. In other 
embodiments, a projection display may be used in conjunction with or in place of the display 

10 1 17. For example, the remote control device 106 may be equipped with a display projection 
apparatus, such that a display can be generated and projected by the device 106, e.g., onto a 
wall or ceiling. The term "display" as used herein is intended to include such projection 
displays as well as other known types of displays. 

It should also be noted that the invention can present information to a user 

1 5 through many other mechanisms in addition to or instead of through the display 1 1 7 or the 
above-noted projection display. For example, intelligent alarms can be presented to a user 
via an audio or speech interface, a tactile interface, or other suitable information 
communication mechanism. An example of a tactile interface is a vibration-type mechanism 
which alerts the user to particular events through certain prespecified types of vibration. 

20 Other examples include a controllable surface that can be altered to form a variety of tactile 
sensations suitable for communicating information, or simply a vibration as a source of an 
alarm. 

Alternatively, the information may be communicated to the user via another 
device such as a television, computer or synthesized speech device. 
25 The communications unit 1 14 of the processor-based device 104 interfaces 

with the set of communications and data devices 108. The set of communications and data 
devices 108 includes a diverse array of information sources such as, e.g., a telephone, a caller 
identification device, a video cassette recorder, a personal computer (PC) connection, a 
World Wide Web (WWW) connection or other Internet connection, a local area network 
30 (LAN) connection, a security system, a satellite system, a terrestrial broadcast system, a cable 
system, a video hard disk (HD) such as local compact disk jukebox, audio or video tape 
library, digital video disk, etc. It should be noted that a single conununication interface 
device could be used to decode information fix)m these and other types of conventional 
media. 
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The communications unit 1 14 also interfaces with the controllable devices 
102. The controllable devices 102 may include at least a subset of the above-noted 
communications and data devices 108, as well as other controllable devices. The controllable 
devices 102 may in some cases therefore be the same as the commxmication and data input 
5 devices 108, e.g., PC, telephones, WWW, satellite, cable, video HD, etc. However, some 
devices 108 provide only input data, e.g., caller ID, TV broadcast, etc., while some devices 
102 are controllable but provide no input data, e.g., lights, X-10, TV chaimel/volume control, 
etc. 

In accordance with the invention, information from one or more of the above- 

10 noted information sources, or from other suitable information sources, is processed in 

accordance with the above-noted control data and filter criteria, and presented to the user via 
the remote control display 117. 

Advantageously, the invention filters the data presented to the user via the 
display 1 1 7 such that the presented information is tailored to learned user preferences. As 

1 S will be apparent to those skilled in the art, the characteristics of the corresponding interfaces 
required in conmiimications unit 1 14 for inter&cing with the set of communications and data 
devices 1 08 will vary depending upon the particular types of information sources used. For 
example, the interfaces in communication imit 1 14 may include, e.g., a satellite, broadcast or 
cable interface, a disk input-output device, or a set of network connections, e.g., transmission 

20 control protocol/Internet protocol (TCP/IP) connections, used to access one or more remote 
servers over the Intemet. 

Input from the communications and data devices 108 or remote control device 
106 may be converted to data using conventional graphical modalities or recognition 
systems. Graphical modalities (such as buttons, sliders, menus, fill-in boxes, etc.) can be 

25 converted to value settings or device commands, for example. Recognition systems for 

handwriting, pen gestures, speech, or visual gestures can also produce summarizing data. As 
an example, a handwritten "Turn TV On," may be recognized as **touched pixels" that are 
then translated to text, converted to control data 118, transmitted to the processor device 104, 
and then issued to a television in a living room via a single IR code. Similarly, visual 

30 gestures, such as signing for the hearing impaired, can be identified by a camera and 
recognition processor so that the same "Turn TV On" is generated. 

It should be noted that elements or groups of elements of the system 100 may 
collectively represent a desktop, laptop or palmtop computer, a personal digital assistant 
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(PDA), a remote control device, a television, or a set-top box, as well as portions or 
combinations of these and other devices. 

It should also be understood that the invention may be implemented in other 
types of systems. For example, in other embodiments of the invention, portions of the 
5 processor-based device 104 may be implemented in a television or a remote control device. 
Moreover, as previously noted, one or more of the elements of the system, such as the 
processor 110, memory 112 and commimications unit 114 may be implemented as an ASIC 
or circuit card to be incorporated into a television, computer, set-top box or other device. 

In operation, the system 100 of FIG. 1 allows the remote control device 106 to 

1 0 receive event-related information from one or more of the communications and data devices 
108 in accordance with stored information regarding learned user preferences. More 
particularly, the system 100 determines from the commimications and data devices 108, the 
occurrence of events likely to be of interest to the user, and provides appropriate 
corresponding event indicators, such as alarms or icons, for presentation to the user via 

1 5 display 1 1 7 or other communications device. These alarms or icons can alert the user that an 
event of interest has occurred, and can specifically identify the area of interest associated 
with the event The user, by entering appropriate filter criteria via the remote control device 
106, can "customize" the device such that the device knows what types of events the user is 
interested in. 

20 Advantageously, the processor-based device 1 04 can associate priorities with 

particular events based on user preference so that events of sufficient priority level can 
interrupt current user activities. For example, the remote control device 106 may be signaled 
via a WWW connection established via one of the conununications and data devices 108. 

In this case, information such as updates from an Internet-based service, e.g., 

25 www , PointCast . com , could be monitored and when determined to be of sufficient 
interest to the user, such as a local school closing, an appropriate alarm, e.g., a particular 
sound, vibration, etc. could be generated by the remote control device 106 in order to alert 
the user as to the occurrence of the event. An identifying icon and corresponding text could 
be displayed on the display 1 17 in conjunction with or in place of the alarm. 

30 The user could then choose to disregard the event, or to take the appropriate 

action to obtain additional information relating to the event. An example of one such 
appropriate action is for the user to enter a command on the remote control device 106 to 
configure the system such that the corresponding WWW input is displayed on the remote 
control display 1 1 7 rather than on a television. The remote control display 1 1 7 in this case 
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serves as an auxiliary display device which extends the available viewing area for icons, so 

that icons need not clutter the television display when a user is watching television. 

« 

As previously noted, the system 100 is configured to learn which events a 
particular user is interested in, and at what times and during what activities the user is 
5 interested in those events. For example, if an incoming telephone call fi-om a telephone 

number associated with a family member occurs, a user might want to be alerted of this event 
regardless of their current activity. Therefore, presentation of a corresponding alami and icon 
via the remote control device 106 may be appropriate. Another type of icon that is displayed 
in this context may be an icon which when activated by a corresponding command entry 

1 0 allows the user to begin recording the television program while taking the call, thereby 
allowing the user to watch any missed portions of the program after the call is concluded. 

Such context-sensitive user behavior can be automatically learned by the 
system over a given period of time, such that the alarms and icons presented to a given user 
intelligently and adaptively reflect previous actions by that user. It is also possible for the 

1 5 system to implement an automated interactive query and programming session with the user 
at particular intervals, e.g., after the system observes several specified actions. 

For example, firom control data 1 1 8 entered on the remote control 106, the 
system may detect that the user muted the television, activated a VCR or other recording 
device, and then answered the telephone when an incoming call was received firom a 

20 particular caller as identified fix>m the caller identification device. The system could 

recognize a sequence of actions of this type and then query the user as to whether the user 
would like to automate such actions when similar conditions occur in the fiiture. In this case, 
the system could also query the user for additional information, e.g., other telephone niunbers 
for which these same actions should occur. These actions would be automatically generated 

25 and sent to the controllable devices 102 via control data 119. Alternatively, the user could 
affirmatively direct the system to automate a given set of actions, e.g., by depressing a 
"leam" key on the remote device 106 after taking the actions. 

As another example, a processing system in accordance vnth the invention 
may be configured to detect when a movie likely to be of interest to a user vnll be broadcast, 

30 based on infomiation supplied by a television reconunender program, and to set ofiT an alarm 
via the remote control device notifying the user before the movie is about to begin. 

As yet another example, the system could alert the user of particular visitors 
when connected to a security system. Using techniques for detecting the presence of a person 
from a video camera data stream and recognizing and cataloging appropriate features of a 
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person for future recognition, the system could alert the user that a particular person has been 
detected. The particular person may be, e.g., a person that the user has had some undesirable 
interaction with in the past. The user can instruct the system to catalog the appropriate 
features of such a person, such that the next time the person is detected by the security system 
5 the authorities can be notified automatically by telephone. 

FIG. 2 is a block diagram illustrating the interaction between the processor- 
based device 104 and a number of additional system elements used to implement the above- 
described functionality. In this embodiment, the additional system elements include a 
filter/rules database 120 and an event log 122. The event log 120 may be any memory 

10 device, e.g., RAM, disk, etc., that is capable of logging event-related information such as the 
time and system state of command and/or control actions or other events that take place in the 
system 100. The filter/rules database 122 may be any memory device capable of storing 
rules and filters that have been established to allow the system to process source data and 
determine if the data should be passed through in unprocessed form, or used to trigger a 

1 5 particular type of processing. 

Although shown as separate from the processor-based device 104 in FIG. 2, 
the elements 120 and 122 may in fact be incorporated into the device 104, e.g., as particular 
portions of the memory 112. The manner in which the processor-based device 104 interacts 
vdth the filter/rules database 120 and the event log 122 is described in conjimction with 

20 FIGS. 3 and 4 below. 

The adaptive engine 1 16 as shown in FIGS. 1 and 2 may represent one or 
more software programs stored in the memory 1 12 and executed by the processor 110. 
Alternatively, the adaptive engine 1 16 may be implemented in firmware, e.g., as an 
"evolvable" circuit on an embedded FPGA that is designed to determine and distinguish user 

25 preferences. 

The adaptive engine 116 may include statistical correlation models of user 
behavior, case-based reasoning, artificial neural networks or other similar learning 
mechanisms. In the illustrative embodiment, the adaptive engine 116 comprises a semi- 
automated event chain query resolution process, i.e., an interactive process, which guides the 
30 system 100 in determining appropriate information presentation and resolving conflicts v^dth 
previously-established rules. The adaptive engine 1 16 in this embodiment includes 
capabilities for event chain adaptation and rule chain trigger event resolution. 

The event chain adaptation capability of the adaptive engine 116 may operate 
as follows. After determining that a particular event chain has terminated, e.g., by methods 
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such as time outs, autocorrelation of event logs, or user directive, an event chain query 
resolution process examines each event in an event log and determines if the pending event 
should be part of a triggering chain. This can be done by simply querying the user with each 
pending event entry. 

5 After the pertinent events have been identified, the user may be asked to 

identify the causal events and the reactive events, and order the two sets of events in either a 
strict sequence, general order, or assert that the sequence is irrelevant. The user may also be 
presented with a list of system states that the user is able to generalize or restrict, as being 
relevant to the event chain being considered, such as time of day, day of week, etc. In this 

1 0 way, a case-based knowledge base can be developed which reflects user preferences. The 
corresponding rules are stored in the filter/rules database 120 for future use in detecting 
particular events, e.g., trigger and filter events. 

The rule chain trigger event resolution as implemented in adaptive engine 116 
may involve the following. Conflicts can arise when a rule is entered into the database or 

1 5 when a trigger event chain is overridden by the user. An example of the latter is a situation in 
which the user allows a telephone answering machine to take a message under the same 
conditions for which the user previously answered the telephone. When these types of 
conflicts are detected, the user may be queried to determine what state information available 
to the system can be specified to delineate the two event chains, resulting in a specialization 

20 of the rules. Other resolution methods can also be used. For example, an old rule may be 
replaced with a new rule in cases in which the two rules are determined to be the same or 
sufficiently similar. 

Detection of rules which are similar when entering rules into the filter/mle 
database may result in further querying of the user to determine if the two rules are in fact 

25 different or if they can be generalized into a single rule. This query process can identify the 
state differences for the user and allow the user to remove them as preconditions or combine 
them. An example of such a case is when similar events lead to the same actions but occur at 
two different times of the day. These same processes could be executed without the 
immediate interaction of the user using similarity-based reasoning or explanation-based 

30 learning but might be more error prone and therefore more fiiistrating to the user. These and 
other strategies could be selected based on marketing or testing decisions. 

FIG. 3 is a data flow diagram showing the operation of the FIG. 1 system in an 
illustrative embodiment of the invention. The data flow shown in this example is 
implemented in the processor-based device 104 and utilizes the filter/rules database 120 and 
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event log 122 of FIG. 2. Incoming control data 118 from the remote control device 106 and 
set of communications and data devices 108 are applied to an input/decision processor 130. 

The signal from the remote control device 106 may be, e.g., a data 
transmission intended to control another device and being passed through the system 100 for 
5 event log processing. The signal from device(s) 108 may be, e.g., any signal from a device 
which acts as an information source with respect to the system 100, Such signals may be in 
the form of, e.g., unprocessed digital or analog media content, processed digital or analog 
media content, control signals, etc. One or more of the device(s) 108 may also act as 
information sinks with respect to the system 100, in that they can receive data from the 

10 system in forms such as, e.g., raw media content input, unprocessed and from outside the 
system, system processed and refined data, control signals, etc. 

The input/decision processor 130 determines, e.g., if the end of an event chain 
has occurred, if the input data is the type that should be recorded in an event recorder 134 
and/or if it should be processed in some way. Event chains may be terminated in any nxmiber 

15 of ways. For example, event chain termination may be based on user-directed input or a 
designated amount of time has elapsed between events. 

The input/decision processor 130 drives a user filter processor 132, the above- 
noted event recorder 134, and an event chain adaptation element 136. The user filter 
processor 132 and trigger event detector 138 operate in accordance with information from the 

20 filter/rules database 120. The user filter processor determines if there is a user-specified 
request in the filter/rules database 120 which matches an event indicated by a given input 
signal. The event recorder 134 records, e.g., system events such as command and/or control 
signals and corresponding detectable system state information. The event chain ad^tation 
element 136 determines, in a manner possibly aided by the user, which events contribute to 

25 an event chain that should trigger some action again in the fiiture without having to be 
directed by the user. 

The output of the user filter processor 1 32 may be applied to a trigger event 
detector 138, which generates an input to an auto-command generator 140 or the user filter 
processor may simply pass on the input. The output of the auto-command generator 140 is 

30 applied as an input to an output processor 142. The trigger event detector 138 determines if 
an incoming signal satisfies the conditions of any rule in the filter/rules database 120 such 
that generated actions should be taken, e.g., all conditions have been satisfied such that some 
stream of control signals should be generated that vdU mute the volume and activate a video 
hard disk or other recording device to record the current program. The auto-command 
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generator 140 generates necessary control data 1 19, e.g., control commands or other signals, 
for controllable devices 102 based on the detection of a trigger event. The output processor 
142 conditions data for output to the remote control device 106 or to one or more of the 
controllable devices 102. 
5 The event recorder 134 processes event-related input received from the 

input/decision processor 130. The event recorder 134 records event information in the event 
log 122, and delivers appropriate output signals to the output processor 142. The contents of 
the event log 122 are used by the event chain adaptation element 1 36. The results of an event 
chain adaptation process are applied to a rule chain trigger event resolution element 144, 

10 which utilizes and updates information from the filter/rules database 120. The rule chain 
trigger event resolution element 144 resolves similar or conflicting rules in filter/rules 
database 120 such that specialized rules can be differentiated from general rules, and 
conflicting rules, if any, can be eliminated or differentiated from each other using additional 
system state information. 

1 S FIG. 4 is a flow diagram illustrating the operations of the processing elements 

of the FIG. 3 data flow diagram. At step ISO, an incoming signal arrives in the input/decision 
processor 130 from either the remote control device 106 or one of the devices in the set of 
devices 108. Step 152 logs the incoming signal and associated information if appropriate, 
i.e., if the incoming signal corresponds to a record event. A record event is an event that 

20 leads to the recording of system-detectable state information, and is typically signaled by a 
system state change. If the incoming signal corresponds to a record event, the event is logged 
in step 1 54 by storing a suitable representation thereof in the event log 122. 

An example of a record event is a signal received from a telephone indicating 
that an incoming call has been detected and the caller ID information for the calling party is 

25 (xxx) xxx-xxxx, which is the telephone number of Mrs. Jones (i.e., the user's mother). This 
information is recorded in the event log 122 for later processing along vsdth other detectable 
system state information. System state information in this example may include, e.g., that a 
pay-per-view (PPV) movie is being viewed on the television 102, a VCR or other recording 
device is not on, audio is being transmitted via the stereo at a volume of 30 dB, and the date 

30 and time. This system state information is useful for rule formation in the filter/rules 

database 120 in case that the same event may subsequently correspond to a trigger event. 

The FIG. 4 process then continues with step 156, which utilizes the filter/rules 
database 120 to determine if the incoming signal corresponds to a trigger event. A trigger 
event is an event which, when combined with all other system state information, satisfies the 
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conditions of a rule in the filter/rules database 120, causing an action to be taken by the 
system automatically, or leading to the system suggesting an action to the user based on a 
confidence factor or user directive. If the incoming signal corresponds to a trigger event, the 
auto-command generator 140 uses the filter/rules database 120 to determine the appropriate 
5 command stream which is generated and dispatched via the output processor 142. 

An example of a trigger event may be the detection of an incoming telephone 
call. The corresponding telephone niunber from caller ID information is checked in the 
filter/rule database 120 and detemiined to be a family member (e.g., the user's mother). This 
event triggers the display of a picture of the user's mother, e.g., on the remote control display 

10 1 1 7, to let the user know who is calling. Other state information available in the system is 
checked against the filter/rules database 120, e.g., state infomiation indicating that the stereo 
is currently on, a television show is being watched, etc. These conditions match rules that 
trigger conunand/control signals to be sent, e.g., to mute the stereo, to suspend display of the 
picture on the television, to activate a video hard disk to record the program, and to begin 

1 5 displaying a video conference connection so that the user can have a video conference with 
the caller. 

Referring again to FIG. 4, if step 1 56 determines that the incoming signal does 
not correspond to a trigger event, the process continues to step 160, which determines if the 
incoming signal corresponds to a filter event. A filter event is one where the user has 

20 requested particular data which can be filtered from some incoming data stream. In this case, 
the user may not want to see the data stream all of the time or may not want to see all of the 
data (e.g., the weather, the stock market) but only when some new information or information 
of a particular type arrives (i.e., show me only these stocks or only show me when these 
stocks go above or below some threshold). 

25 A filter event may trigger other events but does not necessarily generate 

conunand/control signals to other devices, thus distinguishing it from a trigger event. If the 
incoming signal does correspond to a filter event, an event filter process 162 utilizes 
information in the filter/rules database 120 to apply expropriate filtering to the incoming 
information, and if necessary generates an appropriate output to be transmitted by the output 

30 processor 142. 

An example of a filter event may be the receipt of data from a web site, such 
as the above-noted pointcast web site, indicating, e.g., that the weather forecast for a 
particular area of interest, such as the Colorado mountains, is for two feet of snow this 
weekend. Since the user has set this as a filter criterion, residing in the filter/rules database 
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120, an alann and/or icon is displayed or otherwise presented to the user via the remote 
control device 106, TV or other output device. The event filter process 162 generates the 
information needed to direct the presentation of the alarm and/or icon to the user via the 
remote control device 106, TV or other output device and forwards this information to the 
5 output processor 142. 

If step 160 of FIG. 4 determines that the input signal does not correspond to a 
filter event, step 164 determines if the end of an event chain has been detected. If the end of 
an event chain has been detected, the event chain adaptation element 136 utilizes information 
in the filter/rules database 120 to begin event chain adaptation. The rule chain trigger 

10 resolution element 144 then makes the appropriate adjustment(s) to the filter/rules database 
120 based on the adaptation performed in element 136. If step 164 determines that an end of 
event chain has not been detected, the process returns to start (RTS) as shown. The process 
also returns to start after the operations of the rule chain trigger event resolution element 144, 
and after generation of an appropriate output by the output processor 142. 

1 S The processing elements shown in the FIG. 3 data flow diagram, i.e., the 

input/decision processor 130, the user filter processor 132, the event recorder 134, the event 
chain adaptation element 136, the trigger event detector 138, the auto-conmiand generator 
140, the output processor 142 and the rule chain trigger event resolution element 144 may 
each be implemented in whole or in part as one or more software programs stored in memory 

20 112 and executed by the processor 1 10 in the processor-based device 104. 

As previously noted, the processor-based device 104 may be implemented in 
whole or in part using various types of known elements, such as FFGAs or other 
programmable devices, and could be incorporated in whole or in part into the remote control 
device 106, a television, a computer, a set-top box or other system device. 

25 It should also be noted that the processor-based device 1 04 need not be co- 

located with the devices 102, 106, and 108, but may be connected to the system via, e.g., an 
Intemet connection, a telephone connection, or other suitable connection allowing it to 
physically reside elsewhere. In addition, commimications to the remote control device 106 or 
controllable devices 102 may be issued indirectiy through a computer, infirared (IR) interface, 

30 or other appropriate networked device. 

The above-described embodiments of the invention are intended to be 
illustrative only. For example, the invention can be used to implement the presentation of 
many other types of information in conjunction with learned user preferences, in a wide 
variety of different arrangements and configurations, in any desired type of video processing 
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system or device, and in many applications other than those described herein. These and 
numerous other embodiments within the scope of the following claims will be apparent to 
those skilled in the art. 
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CLAIMS: 



1 . An apparatus for use in controlling the presentation of information to a user of 
a system (100), the apparatus comprising: 

a processor-based device (104, 106) operative to automatically determine at 
least one preference of the user with regard to one or more system actions based at least in 
5 part on previous user actions and corresponding system state information, and to initiate 
subsequent execution of at least a subset of the system actions based at least in part on the 
determined preference. 

2. The apparatus of claim 1 wherein the one or more system actions comprises 
10 presentation to the user via a remote control device (106) of a particular type of information 

available from a particular information source (102, 108). 

3. The apparatus of claim 2 wherein the preference of the user is verified by the 
user prior to directing the presentation of information to the user. 

15 

4. The apparatus of claim 1 wherein at least a subset of the user actions are at 
least temporarily stored in at least one other system device and subsequently communicated 
from the other system device to the processor-based device over a transmission medium. 

20 5. The apparatus of claim 4 M^ierein the other system device comprises a remote 

control device (106). 

6. The apparatus of claim 1 wherein a particular set of one or more system 

actions are repeated under the control of the processor-based device when a corresponding 
25 set of one or more conditions are met. 



7. The apparatus of claim 6 wherein the repetition of the one or more system 

actions is implemented automatically without further input from the user. 
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8. The apparatus of claim 6 wherein the repetition of the one or more system 

actions is implemented semi-automatically upon receipt of an appropriate input from the 
user. 

5 9. The apparatus of claim 6 wherein the repetition of the one or more system 

actions is implemented in a user-initiated manner in which execution of each of at least a 
subset of the set of actions occurs only upon receipt of appropriate input from the user. 

10. The apparatus of claim 2 wherein the information source comprises at least 
10 one of a network connection, a telephone connection, and a computer connection. 

11. The apparatus of claim 2 wherein the information source comprises an 
information source associated with at least one of a security system, a satellite system, a 
tenestrial broadcast system, a cable system, and an information storage system. 

15 

12. The apparatus of claim 2 wherein the particular information comprises a 
designated event. 

13. The apparatus of claim 12 wherein the processor-based device determines if 
20 the event comprises at least one of a record event, a trigger event and a filter event, each 

having particular system state information associated therewith. 

14. The apparatus of claim 2 wherein the processor-based device is further 
operative to automatically provide control of at least one additional device based at least in 

25 part on the determined preference. 

15. A method for use in controlling the presentation of information to a user of a 
system (100), the method comprising the steps of: 

automatically determining at least one preference of the user with regard to 
30 one or more system actions based at least in part on previous user actions and corresponding 
system state information; and 

initiating subsequent execution of at least a subset of the system actions based 
at least in part on the determined preference. 
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16. An article of manufacture comprising a machine-readable storage medium 

containing one or more software programs for use in controlling presentation of information 
to a user of a system (100), wherein the programs when executed implement the steps of: 

automatically determining at least one preference of the user with regard to 
one or more system actions based at least in part on previous user actions and corresponding 
system state information; and 

initiating subsequent execution of at least a subset of the system actions based 
at least in part on the determined preference. 
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